{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 不规则张量\n",
    "## 设置"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 概述\n",
    "你的数据有多重形状，你的张量也应该有。不规则张量是嵌套变长列表的张量流等价物，它们使存储和处理非均匀形状的数据变得更加容易，包括：\n",
    "1. 可变长度的特征，如电影中的一组演员。\n",
    "2. 一批可变长度的顺序输入，如句子或视频片段。\n",
    "3. 分层输入，如文本文档，可细分为节、段落、句子和单词。\n",
    "4. 结构化输入中的单个字段，如协议缓冲区。\n",
    "\n",
    "### 能用不规则张量做什么\n",
    "不规则张量支持超过百种的 tensorflow 操作，包括数学操作（比如 tf.add、tf.reduce_mean），数组操作（比如 tf.concat、tf,tile），字符串操作（比如 tf.substr）等："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<tf.RaggedTensor [[6, 4, 7, 4], [], [8, 12, 5], [9], []]>\n",
      "tf.Tensor([2.25              nan 5.33333333 6.                nan], shape=(5,), dtype=float64)\n",
      "<tf.RaggedTensor [[3, 1, 4, 1], [], [5, 9, 2], [6], [], [5, 3]]>\n",
      "<tf.RaggedTensor [[3, 1, 4, 1, 3, 1, 4, 1], [], [5, 9, 2, 5, 9, 2], [6, 6], []]>\n",
      "<tf.RaggedTensor [[b'So', b'lo'], [b'th', b'fo', b'al', b'th', b'fi']]>\n"
     ]
    }
   ],
   "source": [
    "digits = tf.ragged.constant([[3, 1, 4, 1], [], [5, 9, 2], [6], []])\n",
    "words = tf.ragged.constant([[\"So\", \"long\"], [\"thanks\", \"for\", \"all\", \"the\", \"fish\"]])\n",
    "print(tf.add(digits, 3))\n",
    "print(tf.reduce_mean(digits, axis=1))\n",
    "print(tf.concat([digits, [[5, 3]]], axis=0))\n",
    "print(tf.tile(digits, [1, 2]))\n",
    "print(tf.strings.substr(words, 0, 2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "不规则张量也有大量的指定方法和操作，包括工厂方法、转换方法和值的映射操作。可以查看 [tf.ragged](https://tensorflow.google.cn/api_docs/python/tf/ragged) 包了解它支持的操作的列表。\n",
    "\n",
    "和普通张量一样，你可以通过 python 风格的下标去访问不规则张量的指定位置。更多的信息请查看下面的索引（Indexing） 一节。\n",
    "\n",
    "```python\n",
    "print(digits[0]) # 第一列\n",
    "print(digits[:, :2]) # 每一列的前两个值\n",
    "print(digits[:, -2:]) # 每一列的后两个值\n",
    "```\n",
    "\n",
    "就像普通的张量一样，您可以使用 python 算术和比较运算符来执行元素级操作。有关更多信息，参见下面的重载操作符（Overloaded Operators）一节。\n",
    "\n",
    "```python\n",
    "print(digits + 3)\n",
    "print(digits + tf.ragged.constant([[1, 2, 3, 4], [], [5, 6, 7], [8], []]))\n",
    "```\n",
    "\n",
    "如果你需要对一个不规则张量的值完成元素级转换，你可以使用 [tf.ragged.map_flat_values](https://tensorflow.google.cn/api_docs/python/tf/ragged/map_flat_values)，它接受一个函数加上一个或多个参数，并应用这个函数变换不规则张量的值。\n",
    "\n",
    "```python\n",
    "times_two_plus_one = lambda x: x * 2 + 1\n",
    "print(tf.ragged.map_flat_values(times_two_plus_one, digits))\n",
    "```\n",
    "\n",
    "### 构造不规则张量\n",
    "最简单的构建不规则张量的方法是使用 [tf.ragged.constant](https://tensorflow.google.cn/api_docs/python/tf/ragged/constant)，它根据给定的嵌套 python 列表构建不规则张量：\n",
    "```python\n",
    "sentences = tf.ragged.constant([\n",
    "    [\"Let's\", \"build\", \"some\", \"ragged\", \"tensors\", \"!\"],\n",
    "    [\"We\", \"can\", \"use\", \"tf.ragged.constant\", \".\"]])\n",
    "print(sentences)\n",
    "\n",
    "paragraphs = tf.ragged.constant([\n",
    "    [['I', 'have', 'a', 'cat'], ['His', 'name', 'is', 'Mat']],\n",
    "    [['Do', 'you', 'want', 'to', 'come', 'visit'], [\"I'm\", 'free', 'tomorrow']],\n",
    "])\n",
    "print(paragraphs)\n",
    "```\n",
    "\n",
    "不规则张量也可以通过将扁平值张量与行分割张量配对来构造，这些张量指示如何将这些值划分为行，使用工厂类方法，如 [tf.RaggedTensor.from_value_rowids](https://tensorflow.google.cn/api_docs/python/tf/RaggedTensor#from_value_rowids)、[tf.RaggedTensor.from_row_lenges](https://tensorflow.google.cn/api_docs/python/tf/RaggedTensor#from_row_lengths) 和[tf.RaggedTensor.from_row_splits](https://tensorflow.google.cn/api_docs/python/tf/RaggedTensor#from_row_splits)。\n",
    "\n",
    "**tf.RaggedTensor.from_value_rowids**：如果你知道每个值所属于的行，那么你就能使用一个 value_rowids 行分割张量来构建不规则张量：\n",
    "```python\n",
    "print(tf.RaggedTensor.from_value_rowids(\n",
    "    values=[3, 1, 4, 1, 5, 9, 2],\n",
    "    value_rowids=[0, 0, 0, 0, 2, 2, 3]))\n",
    "```\n",
    "\n",
    "**tf.RaggedTensor.from_row_lengths**：如果你知道每个值的列，那么你就能使用一个 row_lengths 列分割张量：\n",
    "```python\n",
    "print(tf.RaggedTensor.from_row_lengths(\n",
    "    values=[3, 1, 4, 1, 5, 9, 2],\n",
    "    row_lengths=[4, 0, 2, 1]))\n",
    "```\n",
    "\n",
    "**tf.RaggedTensor.from_row_splits**：如果你知道每一行开始和结束的索引，那么你能使用一个 row_splits 行分割符：\n",
    "```python\n",
    "print(tf.RaggedTensor.from_row_splits(\n",
    "    values=[3, 1, 4, 1, 5, 9, 2],\n",
    "    row_splits=[0, 4, 4, 6, 7]))\n",
    "```\n",
    "> 要获取工厂方法的列表，参考 [tf.RaggedTensor](https://tensorflow.google.cn/api_docs/python/tf/RaggedTensor)。\n",
    "\n",
    "### 怎么存储不规则张量\n",
    "与普通张量一样，不规则张量中的值必须具有相同的类型，所有的值都必须在相同的嵌套深度(张量的秩)：\n",
    "```python\n",
    "print(tf.ragged.constant([[\"Hi\"], [\"How\", \"are\", \"you\"]]))  # ok: type=string, rank=2\n",
    "\n",
    "print(tf.ragged.constant([[[1, 2], [3]], [[4, 5]]]))        # ok: type=int32, rank=3\n",
    "\n",
    "try:\n",
    "  tf.ragged.constant([[\"one\", \"two\"], [3, 4]])              # bad: multiple types\n",
    "except ValueError as exception:\n",
    "  print(exception)\n",
    "\n",
    "try:\n",
    "  tf.ragged.constant([\"A\", [\"B\", \"C\"]])                     # bad: multiple nesting depths\n",
    "except ValueError as exception:\n",
    "  print(exception)\n",
    "```\n",
    "\n",
    "### 使用样例\n",
    "下面的示例演示了如何使用不规则张量为一批变长查询构造和组合单字符和双字符嵌入，并为每个句子的开头和结尾使用特殊的标记。有关本例中使用操作的更多细节，参见 [tf.ragged](https://tensorflow.google.cn/api_docs/python/tf/ragged) 的文档。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tf.Tensor(\n",
      "[[ 0.01088642 -0.19736682  0.13587682 -0.02848104]\n",
      " [ 0.1769548   0.38557458  0.3999306   0.029276  ]\n",
      " [-0.07602817 -0.12428186  0.07884008 -0.25481457]], shape=(3, 4), dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "queries = tf.ragged.constant([['Who', 'is', 'Dan', 'Smith'],\n",
    "                              ['Pause'],\n",
    "                              ['Will', 'it', 'rain', 'later', 'today']])\n",
    "\n",
    "# 创建一个嵌入表\n",
    "num_buckets = 1024\n",
    "embedding_size = 4\n",
    "embedding_table = tf.Variable(\n",
    "    tf.random.truncated_normal([num_buckets, embedding_size],\n",
    "                       stddev=1.0 / math.sqrt(embedding_size)))\n",
    "\n",
    "# 查找每个单词的嵌入\n",
    "word_buckets = tf.strings.to_hash_bucket_fast(queries, num_buckets)\n",
    "word_embeddings = tf.ragged.map_flat_values(\n",
    "    tf.nn.embedding_lookup, embedding_table, word_buckets)\n",
    "\n",
    "# 在每个句子的开头和结尾加上标记\n",
    "marker = tf.fill([queries.nrows(), 1], '#')\n",
    "padded = tf.concat([marker, queries, marker], axis=1)\n",
    "\n",
    "# 构建单词大图并查找嵌入\n",
    "bigrams = tf.strings.join([padded[:, :-1],\n",
    "                               padded[:, 1:]],\n",
    "                              separator='+')\n",
    "\n",
    "bigram_buckets = tf.strings.to_hash_bucket_fast(bigrams, num_buckets)\n",
    "bigram_embeddings = tf.ragged.map_flat_values(\n",
    "    tf.nn.embedding_lookup, embedding_table, bigram_buckets)\n",
    "\n",
    "# 找出每个句子的平均嵌入\n",
    "all_embeddings = tf.concat([word_embeddings, bigram_embeddings], axis=1) \n",
    "avg_embedding = tf.reduce_mean(all_embeddings, axis=1)\n",
    "print(avg_embedding)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 不规则张量的定义\n",
    "### 参差不齐的规模\n",
    "不规则张量是一个具有一个或多个不规则维的张量，这些维的切片可能有不同的长度。例如，rt=[[3, 1, 4, 1]，[]，[5, 9, 2]，[6]，[]] 的内部(列)维是不规则的，因为列切片 (rt[0，:]，…， rt[4，:]) 有不同的长度。其切片长度相同的维度称为均匀维度。\n",
    "不规则张量的最外层维度总是一致的，因为它由一个切片组成（所以不可能有不同的切片长度）。除了均匀的外层尺寸外，不规则张量也可能具有均匀的内部尺寸。例如，可以使用一个不规则张量（shape [num_sentence， (num_words)， embedding_size]）将每个单词的嵌入存储在一批句子中，其中(num_words) 周围的括号表示维度是不规则的。\n",
    "\n",
    "不规则张量可以有多个不规则维。例如，我们可以使用一个带有形状 [num_documents， (num_paragraph)， (num_sentence)， (num_words)] 的张量来存储一批结构化的文本文档(这里同样使用括号来表示不规则的维度)。\n",
    "\n",
    "**不规则张量形状限制**：\n",
    "不规则张量的形状目前被限制为如下形式：\n",
    "- 单一均匀维度\n",
    "- 随后是一个或多个不规则维度\n",
    "- 然后是零个或多个一致的维度\n",
    "\n",
    "> 注意：这些限制是当前实现的结果，tensorflow 可能在将来放宽它们的限制。\n",
    "\n",
    "### 秩和不规则秩\n",
    "不规则张量中的维数总数称为其秩，不规则张量中的不规则维数称为其不规则秩。在图形执行模式下（即 no-eager 模型），张量的不规则秩在创建时是固定的：它不能依赖于运行时的值，也不能因不同的会话运行而动态变化。一个不规则张量是可能是 [tf.tensor](https://tensorflow.google.cn/api_docs/python/tf/Tensor) 或 [tf.RaggedTensor](https://tensorflow.google.cn/api_docs/python/tf/RaggedTensor) 的值。张量的不规则秩被定义为0。\n",
    "\n",
    "### 不规则张量形状\n",
    "当描述一个不规则张量的形状时，不规则维是用括号括起来的。例如，正如我们在上面看到的，存储一批句子中每个单词的嵌入的 3-D 不规则张量的形状可以写成 [num_sentence， (num_words), embedding_size]。RaggedTensor.shape 属性返回一个 tf.TensorShape，其中不规则维度没有大小。\n",
    "```python\n",
    "tf.ragged.constant([[\"Hi\"], [\"How\", \"are\", \"you\"]]).shape\n",
    "```\n",
    "方法 [tf.RaggedTensor.bounding_shape](https://tensorflow.google.cn/api_docs/python/tf/RaggedTensor#bounding_shape) 可用于查找给定不规则张量的紧边界形状（tight bounding shape）：\n",
    "```python\n",
    "print(tf.ragged.constant([[\"Hi\"], [\"How\", \"are\", \"you\"]]).bounding_shape())\n",
    "```\n",
    "\n",
    "## 不规则张量 VS 稀疏张量\n",
    "一个不规则张量不应该被认为是一个稀疏张量的类型，而应该是一个形状不规则的稠密张量。\n",
    "\n",
    "举一个说明性的例子，考虑一下如何为不规则张量和稀疏张量定义诸如 concat、stack 和 tile 之类的数组操作。连接不整齐的张量连接每一行，形成一个具有组合长度的单行：\n",
    "![title](../img/4_4/ragged_concat.png)\n",
    "```python\n",
    "ragged_x = tf.ragged.constant([[\"John\"], [\"a\", \"big\", \"dog\"], [\"my\", \"cat\"]])\n",
    "ragged_y = tf.ragged.constant([[\"fell\", \"asleep\"], [\"barked\"], [\"is\", \"fuzzy\"]])\n",
    "print(tf.concat([ragged_x, ragged_y], axis=1))\n",
    "```\n",
    "\n",
    "但连接稀疏的张量相当于连接相应的密度张量,如下面的例子所示（Ø表明缺失值）：\n",
    "![title](../img/4_4/sparse_concat.png)\n",
    "```python\n",
    "sparse_x = ragged_x.to_sparse()\n",
    "sparse_y = ragged_y.to_sparse()\n",
    "sparse_result = tf.sparse.concat(sp_inputs=[sparse_x, sparse_y], axis=1)\n",
    "print(tf.sparse.to_dense(sparse_result, ''))\n",
    "```\n",
    "\n",
    "关于为什么这种区别如此重要的另一个例子：考虑一下 tf.reduce_mean 之类的操作的 \"每行平均值\" 的定义。对于一个不规则张量，一行的平均值是行值除以行的宽度的和。但是对于一个稀疏张量来说，一个行的平均值是这个行的值除以稀疏张量的总宽度（它大于或等于最长行的宽度）的和。\n",
    "\n",
    "## 重载操作符\n",
    "RaggedTensor 类重载了标准的Python算术和比较运算符，使得执行基本的元素数学变得容易：\n",
    "```python\n",
    "x = tf.ragged.constant([[1, 2], [3], [4, 5, 6]])\n",
    "y = tf.ragged.constant([[1, 1], [2], [3, 3, 3]])\n",
    "print(x + y)\n",
    "```\n",
    "\n",
    "由于重载操作符执行元素计算，所以所有二进制操作的输入必须具有相同的形状，或者可以广播为相同的形状。在最简单的广播情况下，单个标量与不规则张量中的每个值元素级组合:\n",
    "```python\n",
    "x = tf.ragged.constant([[1, 2], [3], [4, 5, 6]])\n",
    "print(x + 3)\n",
    "```\n",
    "\n",
    "有关更高级的案例的讨论，参阅关于广播（Broadcasting）的部分。\n",
    "\n",
    "不规则张量重载与普通张量相同的一组运算符：一元运算符-、~ 和 abs(); 和二进制运算符 +、-、*、/、/ /、%、* *、&、|、^、==、<、<=、>、>=。\n",
    "\n",
    "## 索引\n",
    "不规则张量支持python风格的索引，包括多维索引和切片。下面的例子演示了二维和三维不规则张量的不规则张量索引：\n",
    "### 索引一个一维不规则张量\n",
    "```python\n",
    "queries = tf.ragged.constant(\n",
    "    [['Who', 'is', 'George', 'Washington'],\n",
    "     ['What', 'is', 'the', 'weather', 'tomorrow'],\n",
    "     ['Goodnight']])\n",
    "print(queries[1])\n",
    "\n",
    "print(queries[1, 2])                # 一个词\n",
    "\n",
    "print(queries[1:])                  # 除了第一行\n",
    "\n",
    "print(queries[:, :3])               # 每个查询的前3个单词\n",
    "\n",
    "print(queries[:, -2:])              # 每个查询的最后两个单词\n",
    "```\n",
    "\n",
    "### 索引具有两个不规则维的三维不规则张量\n",
    "```python\n",
    "rt = tf.ragged.constant([[[1, 2, 3], [4]],\n",
    "                         [[5], [], [6]],\n",
    "                         [[7]],\n",
    "                         [[8, 9], [10]]])\n",
    "\n",
    "print(rt[1])                        # 第二行（2-D 不规则张量）\n",
    "\n",
    "print(rt[3, 0])                     # 第四行第一元素（一维张量）\n",
    "\n",
    "print(rt[:, 1:3])                   #每行 1-3 项（3-D 不规则张量）\n",
    "\n",
    "print(rt[:, -1:])                   # 每一行的最后一项（3-D 不规则张量）\n",
    "```\n",
    "\n",
    "不规则支持多维索引和切片，但有一个限制：不允许索引到不规则的维度。这种情况是有问题的，因为指示的值可能存在于某些行中，而不是其他行中。在这种情况下，我们是否应该（1）引发索引器错误；（2）使用默认值；或者（3）跳过该值并返回一个行数少于开始时的张量，这并不明显。遵循 python 的指导原则（\"面对模棱两可的情况，拒绝猜测的诱惑\"），我们目前不允许这种操作。\n",
    "\n",
    "## 张量类型转换\n",
    "RaggedTensor 张量类定义了可用于在 RaggedTensors 和 tf.Tensors 或 tf.SparseTensors 之间转换的方法：\n",
    "```python\n",
    "ragged_sentences = tf.ragged.constant([\n",
    "    ['Hi'], ['Welcome', 'to', 'the', 'fair'], ['Have', 'fun']])\n",
    "print(ragged_sentences.to_tensor(default_value=''))\n",
    "\n",
    "print(ragged_sentences.to_sparse())\n",
    "\n",
    "x = [[1, 3, -1, -1], [2, -1, -1, -1], [4, 5, 8, 9]]\n",
    "print(tf.RaggedTensor.from_tensor(x, padding=-1))\n",
    "\n",
    "st = tf.SparseTensor(indices=[[0, 0], [2, 0], [2, 1]],\n",
    "                     values=['a', 'b', 'c'],\n",
    "                     dense_shape=[3, 3])\n",
    "print(tf.RaggedTensor.from_sparse(st))\n",
    "```\n",
    "\n",
    "## 评估不规则张量\n",
    "### eager execution\n",
    "在 eager execution 模式下，会立即计算不规则张量。要访问它们包含的值，你可以:\n",
    "- 使用 [tf.RaggedTensor.to_list()](https://tensorflow.google.cn/api_docs/python/tf/RaggedTensor#to_list) 方法，该方法将不规则张量转换为 python 列表：\n",
    "```python\n",
    "rt = tf.ragged.constant([[1, 2], [3, 4, 5], [6], [], [7]])\n",
    "print(rt.to_list())\n",
    "```\n",
    "- 使用 python 索引。如果你选择的张量不包含不规则维，那么它将以 EagerTensor 的形式返回。然后可以使用 numpy() 方法直接访问该值：\n",
    "```python\n",
    "print(rt[1].numpy())\n",
    "```\n",
    "- 使用 [tf.ragged tensor.values](https://tensorflow.google.cn/api_docs/python/tf/RaggedTensor#values) 和 [tf.RaggedTensor.row_splits](https://tensorflow.google.cn/api_docs/python/tf/RaggedTensor#row_splits) 属性或诸如[tf.RaggedTensor.row_lenges()](https://tensorflow.google.cn/api_docs/python/tf/RaggedTensor#row_lengths) 和 [tf.RaggedTensor.value_rowids()](https://tensorflow.google.cn/api_docs/python/tf/RaggedTensor#value_rowids) 等行划分方法将不规则张量分解为其组件：\n",
    "```python\n",
    "print(rt.values)\n",
    "\n",
    "print(rt.row_splits)\n",
    "```\n",
    "\n",
    "### 广播\n",
    "广播是使不同形状的张量对元素运算具有相容的形状的过程。有关广播的更多背景资料，参阅：\n",
    "- [Numpy: Broadcasting](https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html)\n",
    "- [tf.broadcast_dynamic_shape](https://tensorflow.google.cn/api_docs/python/tf/broadcast_dynamic_shape)\n",
    "- [tf.broadcast_to](https://tensorflow.google.cn/api_docs/python/tf/broadcast_to)\n",
    "\n",
    "广播两个输入 x 和 y 的基本步骤是：\n",
    "1. 如果 x 和 y 没有相同数量的维度，那么添加外部维度（大小为）)，直到它们有相同数量的维度。\n",
    "2. 对于 x 和 y 大小不同的每个维度：\n",
    "    - 如果 x 或 y 在维度 d 中大小为 1，则跨维度 d 重复其值以匹配其他输入的大小。\n",
    "    - 否则，引发异常（x 和 y 不兼容广播）。\n",
    "\n",
    "其中一个张量在一个均匀维中的大小是一个单一的数（该维上的切片的大）；一个不规则维度中的张量的大小是一个切片长度的列表（对于该维度上的所有切片）。\n",
    "\n",
    "**广播的例子**：\n",
    "```python\n",
    "# x       (2D 不规则张量):  2 x (行数)\n",
    "# y       (标量)\n",
    "# 结果  (2D 不规则张量):  2 x (行数)\n",
    "x = tf.ragged.constant([[1, 2], [3]])\n",
    "y = 3\n",
    "print(x + y)\n",
    "\n",
    "# x         (2d 不规则张量):  3 x (行数)\n",
    "# y         (2d 张量):  3 x          1\n",
    "# 结果    (2d 不规则张量):  3 x (行数)\n",
    "x = tf.ragged.constant(\n",
    "   [[10, 87, 12],\n",
    "    [19, 53],\n",
    "    [12, 32]])\n",
    "y = [[1000], [2000], [3000]]\n",
    "print(x + y)\n",
    "\n",
    "# x      (3d 不规则张量):  2 x (r1) x 2\n",
    "# y      (2d 不规则张量):         1 x 1\n",
    "# 结果 (3d 不规则张量):  2 x (r1) x 2\n",
    "x = tf.ragged.constant(\n",
    "    [[[1, 2], [3, 4], [5, 6]],\n",
    "     [[7, 8]]],\n",
    "    ragged_rank=1)\n",
    "y = tf.constant([[10]])\n",
    "print(x + y)\n",
    "\n",
    "# x      (3d 不规则张量):  2 x (r1) x (r2) x 1\n",
    "# y      (1d 张量):                    3\n",
    "# 结果 (3d 不规则张量):  2 x (r1) x (r2) x 3\n",
    "x = tf.ragged.constant(\n",
    "    [\n",
    "        [\n",
    "            [[1], [2]],\n",
    "            [],\n",
    "            [[3]],\n",
    "            [[4]],\n",
    "        ],\n",
    "        [\n",
    "            [[5], [6]],\n",
    "            [[7]]\n",
    "        ]\n",
    "    ],\n",
    "    ragged_rank=2)\n",
    "y = tf.constant([10, 20, 30])\n",
    "print(x + y)\n",
    "\n",
    "################以下是一些不会传播的形状的例子:#####################\n",
    "\n",
    "# x      (2d 不规则张量): 3 x (r1)\n",
    "# y      (2d 张量): 3 x    4  # 尾部尺寸不匹配\n",
    "x = tf.ragged.constant([[1, 2], [3, 4, 5, 6], [7]])\n",
    "y = tf.constant([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]])\n",
    "try:\n",
    "  x + y\n",
    "except tf.errors.InvalidArgumentError as exception:\n",
    "  print(exception)\n",
    "\n",
    "# x      (2d 不规则张量): 3 x (r1)\n",
    "# y      (2d 不规则张量): 3 x (r2)  # 不规则张量维度不匹配\n",
    "x = tf.ragged.constant([[1, 2, 3], [4], [5, 6]])\n",
    "y = tf.ragged.constant([[10, 20], [30, 40], [50]])\n",
    "try:\n",
    "  x + y\n",
    "except tf.errors.InvalidArgumentError as exception:\n",
    "  print(exception)\n",
    "\n",
    "# x      (3d 不规则张量): 3 x (r1) x 2\n",
    "# y      (3d 不规则张量): 3 x (r1) x 3  # 尾部尺寸不匹配\n",
    "x = tf.ragged.constant([[[1, 2], [3, 4], [5, 6]],\n",
    "                        [[7, 8], [9, 10]]])\n",
    "y = tf.ragged.constant([[[1, 2, 0], [3, 4, 0], [5, 6, 0]],\n",
    "                        [[7, 8, 0], [9, 10, 0]]])\n",
    "try:\n",
    "  x + y\n",
    "except tf.errors.InvalidArgumentError as exception:\n",
    "  print(exception)\n",
    "```\n",
    "\n",
    "## RaggedTensor 编码\n",
    "不规则张量是用 RaggedTensor 类来编码的。在内部，每个 RaggedTensor 包括：\n",
    "- 一个值张量，它把可变长度的行连接成一个扁平的列表。\n",
    "- 一个 row_splits，它指示那些平展的值如何划分为行。特别是，行 rt[i] 的值存储在切片 rt.values[rt.row_splits[i]:rt.row_splits[i+1]]。\n",
    "![title](../img/4_4/ragged_encoding.png)\n",
    "```python\n",
    "rt = tf.RaggedTensor.from_row_splits(\n",
    "    values=[3, 1, 4, 1, 5, 9, 2],\n",
    "    row_splits=[0, 4, 4, 6, 7])\n",
    "print(rt)\n",
    "```\n",
    "\n",
    "### 多个不规则维度\n",
    "一个具有多个不规则维的不规则张量通过对值张量使用嵌套的 RaggedTensor 进行编码。每个嵌套的 RaggedTensor 都增加了一个单独的不规则维：\n",
    "![title](../img/4_4/ragged_rank_2.png)\n",
    "```python\n",
    "rt = tf.RaggedTensor.from_row_splits(\n",
    "    values=tf.RaggedTensor.from_row_splits(\n",
    "        values=[10, 11, 12, 13, 14, 15, 16, 17, 18, 19],\n",
    "        row_splits=[0, 3, 3, 5, 9, 10]),\n",
    "    row_splits=[0, 1, 1, 5])\n",
    "print(rt)\n",
    "print(\"形状: {}\".format(rt.shape))\n",
    "print(\"不规则维数: {}\".format(rt.ragged_rank))\n",
    "```\n",
    "工厂函数 [tf.RaggedTensor.from_nested_row_splits](https://tensorflow.google.cn/api_docs/python/tf/RaggedTensor#from_nested_row_splits) 可用于通过提供 row_splits 张量列表，直接构造具有多个不规则维度的 RaggedTensor：\n",
    "```python\n",
    "rt = tf.RaggedTensor.from_nested_row_splits(\n",
    "    flat_values=[10, 11, 12, 13, 14, 15, 16, 17, 18, 19],\n",
    "    nested_row_splits=([0, 1, 1, 5], [0, 3, 3, 5, 9, 10]))\n",
    "print(rt)\n",
    "```\n",
    "\n",
    "### 统一的内部维度\n",
    "内维数一致的不规则张量用多维 tf.Tensor 编码。\n",
    "![title](../img/4_4/uniform_inner.png)\n",
    "```python\n",
    "rt = tf.RaggedTensor.from_row_splits(\n",
    "    values=[[1, 3], [0, 0], [1, 3], [5, 3], [3, 3], [1, 2]],\n",
    "    row_splits=[0, 3, 4, 6])\n",
    "print(rt)\n",
    "print(\"Shape: {}\".format(rt.shape))\n",
    "print(\"Number of ragged dimensions: {}\".format(rt.ragged_rank))\n",
    "```\n",
    "### 替代 row-partitioning 的方案\n",
    "RaggedTensor 类使用 row_splits 作为主要机制来存储有关如何将值分区为行的信息。不过，RaggedTensor 还提供了对四种可选行分区方案的支持，根据数据的格式，使用这些方案更方便。在内部，RaggedTensor 使用这些附加方案来提高某些情况下的效率：\n",
    "\n",
    "**Row lengths**：' row_length '是一个带有形状 `[nrows` 的向量，它指定了每一行的长度。\n",
    "\n",
    "**Row starts**：' row_starts '是一个形状为 `[nrows]` 的向量，它指定每一行的起始偏移量。相当于 \"row_splits (: 1)\"。\n",
    "\n",
    "**Row limits**：' row_limits '是一个带有形状 ` [nrows]` 的向量，它指定每一行的停止偏移量。相当于 \"row_splits [1:]\"。\n",
    "\n",
    "**Row indices and number of rows**：' value_rowids '是一个形状 `[nvals]` 的向量，与值一一对应，指定每个值的行索引。特别是，行 \"rt[row]\" 由值 \"rt\" 组成。其中'value_rowids[j]==row'。\"nrows\" 是一个整数，指定 RaggedTensor 中的行数。尤其是，\"nrows\" 用于指示后面的空行。\n",
    "\n",
    "例如，下面的不规则张量是等价的：\n",
    "```python\n",
    "values = [3, 1, 4, 1, 5, 9, 2, 6]\n",
    "print(tf.RaggedTensor.from_row_splits(values, row_splits=[0, 4, 4, 7, 8, 8]))\n",
    "print(tf.RaggedTensor.from_row_lengths(values, row_lengths=[4, 0, 3, 1, 0]))\n",
    "print(tf.RaggedTensor.from_row_starts(values, row_starts=[0, 4, 4, 7, 8]))\n",
    "print(tf.RaggedTensor.from_row_limits(values, row_limits=[4, 4, 7, 8, 8]))\n",
    "print(tf.RaggedTensor.from_value_rowids(\n",
    "    values, value_rowids=[0, 0, 0, 0, 2, 2, 2, 3], nrows=5))\n",
    "```\n",
    "\n",
    "RaggedTensor 类定义了可以用来构造这些行划分张量的方法：\n",
    "```python\n",
    "rt = tf.ragged.constant([[3, 1, 4, 1], [], [5, 9, 2], [6], []])\n",
    "print(\"      values: {}\".format(rt.values))\n",
    "print(\"  row_splits: {}\".format(rt.row_splits))\n",
    "print(\" row_lengths: {}\".format(rt.row_lengths()))\n",
    "print(\"  row_starts: {}\".format(rt.row_starts()))\n",
    "print(\"  row_limits: {}\".format(rt.row_limits()))\n",
    "print(\"value_rowids: {}\".format(rt.value_rowids()))\n",
    "```\n",
    "\n",
    "注意：tf.RaggedTensor.values 和 tf.RaggedTensors.row_splits 是属性，而其余的行分区访问器都是方法。这反映了行分割是主要的底层表示，必须计算其他行分割张量。\n",
    "\n",
    "不同的行分区方案的一些优点和缺点是：\n",
    "- **高效索引**：row_split、row_started 和 row_limits 方案都支持将常量时间索引转换为不规则张量。value_rowids 和 row_length 方案则没有。\n",
    "- **较小的编码大小**：value_rowids 方案在存储有大量空行的不规则张量时更有效，因为张量的大小只取决于值的总数。另一方面，其他四种编码在存储具有较长行数的不规则张量时更有效，因为它们每一行只需要一个标量值。\n",
    "- **高效连接**：row_length方案在连接不规则张量时更有效，因为当两个张量连接在一起时，行长度不会改变(但是行分割和行索引会改变)。\n",
    "- **兼容性**：value_rowids 方案与 tf.segment_sum 等操作使用的分段格式（[segmentation format](https://tensorflow.google.cn/api_docs/python/tf/math#about_segmentation)）匹配。row_limits 方案与 [tf.sequence_mask](https://tensorflow.google.cn/api_docs/python/tf/sequence_mask) 等操作使用的格式相匹配。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6rc1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
